Portable information device having real-time display with relevant information

ABSTRACT

A portable information device including a body, a position information portion that is provided in the body and that outputs position information, an image sensor that is provided in the body, a display system that is provided in the body and that displays, on a local field in a screen field, relevant information based on the position information such that the displayed relevant information is overlapped on a real-time image, an input portion that is provided on the body and, by use of which user operates an icon, the icon being displayed on the local field such that the displayed icon is overlapped on the real-time image and an imaging button that is provided, separately from the input portion, on the body and that is operated by user to capture a subject image via the image sensor.

CROSS-REFERENCE TO RELATED APPLICATIONS

This is a divisional of U.S. application Ser. No. 17/142,722 filed onJan. 6, 2021, which is a divisional of Ser. No. 15/648,777 filed on Jul.13, 2017, which is a divisional of U.S. application Ser. No. 14/247,737filed Apr. 8, 2014, which is a divisional of U.S. application Ser. No.12/461,324 filed Aug. 7, 2009, which is a non-provisional applicationclaiming priority to and the benefit of U.S. provisional application No.61/136,294, filed Aug. 26, 2008, and claims priority to Japanese PatentApplication No. 2008-205729, filed Aug. 8, 2008, the disclosures ofwhich are hereby incorporated herein by reference in their entireties.

BACKGROUND Field of the Invention

The present invention relates to a portable information device, imagingapparatus, and information acquisition system.

Description of Related Art

Conventionally, there are portable information devices such as imagingapparatuses (for example, digital cameras), which have an imagingfunction, having a function for detecting the position information ofthe portable information devices and processing image data, which isacquired by performing imaging, in association with the detectedposition information (for example, see Japanese Patent ApplicationPublication No. 2004-15187).

A purpose of some aspects of the present invention is to provide aportable information device, an imaging apparatus, and an informationacquisition system, that can enhance a user's activity.

SUMMARY

According to an aspect of the present invention, a portable informationdevice is provided, the portable information device including: an imagesensor; a position information acquisition portion that outputs positioninformation; an orientation sensor; and a display that displays an imageof an object out put from the image sensor and displays relevantinformation on the object, the relevant information being obtained basedon at least measurement data from the position information acquisitionportion and the orientation sensor.

According to another aspect of the present invention, an informationacquisition system is provided, the information acquisition systemincluding: a server; a database; and a portable information device thatcomprises an image sensor, a position information acquisition portionthat outputs position information, an orientation sensor, a display, anda wireless communication portion, the display displaying an image of anobject output from the image sensor and displays relevant information onthe object, the relevant information being obtained from the databasevia the wireless communication portion and the server based on at leastmeasurement data from the position information acquisition portion andthe orientation sensor.

According to another aspect of the present invention, an imagingapparatus is provided, the imaging apparatus including: an imagingportion that takes an image of a subject; a position informationacquisition portion that acquires position information of an imagingposition; and an orientation information acquisition portion thatacquires orientation information representing an imaging direction atthe imaging position.

According to another aspect of the present invention, an informationacquisition is provided, the information acquisition system including:an imaging apparatus; and an information search system. The informationsearch system searches for the information on a subject of which animage is taken by the imaging apparatus and transmits the search resultto the imaging apparatus. The imaging apparatus includes: an imagingportion that takes an image of a subject; a position informationacquisition portion that acquires position information of an imagingposition; and an orientation information acquisition portion thatacquires orientation information representing an imaging direction atthe imaging position. The information search system includes: a databasein which the information on the subject is stored in association withthe position information and the orientation information; and aninformation search server that searches for the information on thesubject corresponding to the position information and the orientationinformation acquired by performing imaging for image data from thedatabase based on the position information and the orientationinformation of the image data acquired by performing imaging using theimaging apparatus.

According to another aspect of the present invention, a portableinformation device is provided, the portable information deviceincluding; a position information acquisition portion that outputsposition information; a display; and a control portion that displaysinformation on an object and an image of the object so as to beassociated with each other, the information on the object being obtainedbased on at least the position information from the position informationacquisition portion, and on data used for identifying the object that isimaged at an imaging position, the imaging position being on the basisof the position information.

According to some aspects of the present invention, a portableinformation device, an imaging apparatus, and an information acquisitionsystem, that can enhance a user's activity, are provided.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram representing a configuration example of aninformation acquisition system according to an embodiment of the presentinvention.

FIG. 2 is a schematic diagram representing a configuration example of abuilding structure table that is stored in a database represented inFIG. 1.

FIG. 3 is a flowchart representing the flow of an information searchingprocess for an image of which the image is taken by a digital cameraaccording to this embodiment.

FIG. 4 is a diagram representing an example of a display image displayedin the digital camera according to this embodiment.

FIG. 5 is a diagram representing an example of a display image displayedin the digital camera according to this embodiment.

FIG. 6 is a diagram representing an example of a display image displayedin the digital camera according to this embodiment.

FIG. 7 is a flowchart representing the flow of an information searchingprocess for an image being taken by the digital camera according to thisembodiment.

FIG. 8 is a flowchart representing the flow of an information searchingprocess on the basis of the store information that is input to thedigital camera according to this embodiment.

DESCRIPTION OF EMBODIMENTS

Hereinafter, an embodiment of the present invention will be describedwith reference to the accompanying drawings. FIG. 1 is a block diagramrepresenting a configuration example of a portable information device(an imaging apparatus) and the information acquisition system accordingto the embodiment.

In FIG. 1, the information acquisition system includes a digital camera1 as an example of the imaging apparatus and an information searchsystem 2. The digital camera 1 transmits or receives data to or from theinformation search system 2 (or an information search server 21 to bedescribed later) through a wireless base station 3 and an informationcommunication network 4 such as the Internet by performing datacommunication with the wireless base station 3 through wirelesscommunication. The imaging apparatus according to this embodiment is animaging unit-equipped mobile terminal such as the above-describeddigital camera 1 or a camera-equipped cellular phone.

The digital camera 1 has a browsing function. The digital camera 1includes a control portion 11, a communication portion 12, an imagingportion 13, a GPS (global positioning system) (a position informationacquisition portion) locator 14, a memory portion 15, an orientationsensor 16, a display portion 17, and a timer 18. Here, the wirelesscommunication according to this embodiment includes communication usingelectric waves, light, sound waves, or the like without using any wireas a transmission channel.

The imaging portion 13 includes a CCD (charge coupled device) imagesensor or a CMOS (complementary metal oxide semiconductor) image sensor,a lens, and the like. The imaging apparatus 13 takes an image of asubject (an object of which image is to be taken) and outputs image datato the control portion 11. In the description below, a subject accordingto this embodiment represents a building structure (a commercialbuilding such as a store, a public building such as a school or ahospital, a factory, a house such as a mansion, an artificialconstruction such as a steel tower, a bridge, a dam, an amusement park,a statue, or the like).

When receiving a control signal (for example, a control signal output bya peripheral circuit, not shown in the figure, that has detected thepressing of an imaging button B1) from a user for directing to take animage or a control signal (for example, a control signal output by aperipheral circuit, not shown in the figure, that has detected theselection of an icon (a signage) by the user touching the icon thatrepresents a search or the like) for directing an information search,the GPS locator 14 acquires information on the latitude and longitude ofthe digital camera 1 and outputs the information to the control portion11.

The orientation sensor 16 is configured by an electronic compass (ageomagnetic sensor) or the like. When receiving a control signal (forexample, a control signal output by a peripheral circuit, not shown inthe figure, that has detected the press of an imaging button B1) from auser for directing to take an image or a control signal (for example, acontrol signal output by a peripheral circuit, not shown in the figure,that has detected the selection of an icon (a signage) by the usertouching the icon that represents a search or the like) for directingthe information search, the orientation sensor 16 detects theorientation, for example, of the serial direction in which the CCD andthe lens of the imaging portion 13 are disposed and outputs the detectedorientation of the serial direction to the control portion 11 as theorientation information. Alternatively, the orientation sensor 16 can beconfigured to detect the orientation of the direction toward the subject(that is, the imaging direction) viewed from the user and output theorientation information of the subject to the control portion 11. Here,when an orientation angle is used as the orientation information, theorientation angle is represented in units of degrees, minutes andseconds. For example, the orientation angle of the north side is used asa reference of 0 degree, and the orientation angles of the east side,the south side, and the west side are denoted by 90 degrees, 180degrees, and 270 degrees in the clockwise direction from the north sidewith the latitude and the longitude of the position information used asa center point.

When a control signal for directing to take an image is received from aperipheral circuit not shown in the figure by the user's pressing of theimaging button B1 or the like, the control portion 11 assigns imageidentification information to the picked up image data and writes thepickup up image data, time data acquired from the timer 18, the positioninformation, and the orientation information in the memory portion 15 inassociation with each image identification information, in the order ofimaging.

The display portion 17 displays the image data that is taken by theimaging portion 13 or the image data that is selected by the user as animage to be displayed and is read out from the memory portion 15 by thecontrol portion 11.

In addition, the display portion 17, as described below, displays theimage data received from the information search system 2, text data(character data) input by the user, or the like under the control of thecontrol portion 11.

The communication portion 12 transmits or receives data such as imagedata, text data, a control signal, or the like to or from theinformation search system 2 through the information communicationnetwork 4 by transmitting or receiving data to or from the wireless basestation 3 by using wireless communication.

The information search system 2, as shown in FIG. 1, includes theinformation search server 21 and a database 22. The database 22 can beset up in a memory medium (for example, a memory, an HDD, or the like)inside the information search server 21, a memory medium that isexternally attached, or a memory medium of another terminal.

In the database 22, as a building structure table represented in FIG. 2,building structure identification information used for identifyingbuilding structures, building names as the names of the buildingstructures, building structure information (information such as anaddress, a telephone number, a type, and information on peripheral imagedata of the building structure), the position information such as thelatitude and the longitude of the building structure, descriptions ofthe building structures (information prepared by a store for a casewhere the building structure is the store), and post information (acomment such as evaluation of a visited user, image data posted by theuser, or the like) are stored.

In addition, in the database 22, the map data, in which the registeredbuilding structures are disposed with the latitude and the longitude setas the coordinate axes in a two-dimensional plane, is stored. Here, inthe map data, each building structure is represented in a position atthe latitude and the longitude corresponding to the building structureidentification information.

The information search server 21 searches for a building structure thatis closest from the position at the latitude and the longituderepresented by the position information in the direction represented bythe orientation information by using the position information and theorientation information described above which have been input andacquires the building structure identification information of the foundbuilding structure.

In addition, the information search server 21 searches for a buildingstructure corresponding to the building structure identificationinformation of the found and acquired building structure from thebuilding structure table and transmits the information of the foundbuilding structure (the building structure name, the building structureinformation, and the like) to the digital camera 1. Alternatively, theinformation search server 21 can be configured to selectively transmitthe information of the building structure to be transmitted to thedigital camera 1, for example, depending on the need for the capacity ofthe transmission data or the reception data or the like. In such a case,the information search server 21 can be configured to retransmit theremaining information out of the information of the building structureto the digital camera 1, for example, in accordance with a predeterminedoperation (for example, a request from the digital camera 1) or thelike.

<Searching for Information on Imaging By Digital Camera 1>

Next, the operation of this embodiment will be described with referenceto FIGS. 1 and 3 together with FIGS. 4 to 6. FIG. 3 is a flowchartrepresenting an example of an information searching process according tothis embodiment for a case where information search is desired to beperformed in a state in which the digital camera 1 is in the process ofimaging a building structure. FIGS. 4 to 6 are diagrams representing thedisposition of a screen displayed in the display portion 17 of thedigital camera 1 and the imaging button B1 of the digital camera 1.

In description below, as an example, a case where a user searches for adining restaurant while walking in a downtown street will be described.

When the user finds a restaurant having an external view of whichinformation is desired to be acquired with the lens of the digitalcamera 1 positioned toward the restaurant having the favorable exteriorview while strolling in the downtown street (Step S1), as shown in FIG.4, the user touches (presses down) a browsing icon 11 that is set up inthe image display field MP of the digital camera 1, thereby the browsingicon 11 is selected (Step S2).

At this moment, the control portion 11 displays image data, of which theimage is taken by the imaging portion 13, that is input from the imagingportion 13 in the imaging state, that is, the image data of a restaurantof which the image is taken, as shown in FIG. 4, in the image displayfield MP through the display portion 17. In addition, the image displayfield MP (image display portion) according to this embodiment isconfigured by a touch panel (for example, including a pressure-sensitivetype, an electrostatic type, or a sound wave type).

In the above-described Step S2, the browsing icon 11 can be configuredto be selected by being touched after the imaging button B1 of thedigital camera 1 is pressed down, while the image display field MP inwhich the image of the restaurant is displayed is viewed.

By touching the browsing icon 11, a signal indicating the selection ofthe browsing icon 11 is output from a peripheral circuit not shown inthe figure to the control portion 11.

Then, the control portion 11 detects the selection of the browsing icon11 in accordance with the input signal indicating the selection of theinput browsing icon 11, assigns image identification information to theimage data of the above-described restaurant, the image of which istaken by the imaging portion 13, and writes the image data and theimaging time in the memory portion 15 in association with the imageidentification information for storing the image data (Step S3).

In addition, when the browsing icon 11 is touched, the control portion11 writes the position information acquired by the GPS locator 14 andthe orientation information acquired by the orientation sensor 16 in thememory portion 15 in association with the image identificationinformation as related information (Step S4).

Then, after writing the image data and the related information in thememory portion 15, the control portion 11 transmits a search requestsignal that includes the camera identification information (useridentification information assigned to a user or identificationinformation assigned to a digital camera 1) for identifying the digitalcamera 1, the position information, and the orientation information tothe information search system 2 through the communication portion 12(Step S5). Alternatively, the control portion 11 can be configured totransmit the search request signal to the information search system 2through the communication portion 12 without writing the image data, therelated information, and the like in the memory portion 15 in theabove-described Steps S3 to S5.

Next, when the server 21 of the information search system 2 receives theabove-described search request signal from the digital camera 1, theserver 21 extracts a building structure of a search target from the mapdata of the database 22 based on the position information and theorientation information that are included in the search request signal.Then, the server 21 transmits each type of information on the buildingstructure table, which is included in the building structure tablerepresented in FIG. 2, including the building structure identificationinformation to the digital camera 1 as search result informationincluding the building structure identification information (Step S7).As an example, at this moment, the server 21 reads out the address ofthe digital camera 1 on the network which is added for a case where thesearch request signal is transmitted from the digital camera 1 throughthe communication portion 12 and transmits the search result informationtowards this address. Alternatively, in the above-described Step S7, theserver 12 can be configured to extract the building structure of thesearch target from the map data of the database 22 based on the positioninformation and the orientation information, which are included in thesearch request signal, only in a case where registration of the cameraidentification information included in the search request signal in theabove-described user registration table that is stored in the database22 is detected. In such a case, when the camera identificationinformation included in the search request signal is detected not to beregistered in the above-described user registration table, it ispreferable that the server 21 transmits information indicating the needfor user registration, for example, on the basis of the cameraidentification information to the digital camera 1.

Then, when the control portion 11 of the digital camera 1 receives theabove-described search result information from the information searchsystem 2 through the communication portion 12, as shown in FIG. 5, thecontrol portion 11 displays the building information (relevantinformation on the object) in the information display field SP throughthe display portion 17 (Step S8). For example, when the buildingstructure is a restaurant, the information of the type of the restaurant(Chinese, Japanese, French. Italian, or the like) and a telephonenumber, an address, and the like of the restaurant are displayed. Inaddition, the information display field SP represented in FIG. 5 is anexample according to this embodiment, and the information display fieldSP can be displayed such that a part thereof is overlapped with theimage of the restaurant. In addition, the information display field SPis a part of the image display field MP. The information display fieldSP is displayed, for example, by touching the above-described browsingicon 11, a CM icon 12 to be described later, or the like for theselection.

In addition, when the user touches the CM icon 12, the above-describedperipheral circuit outputs a signal indicating the selection of the CMicon 12 to the control portion 11.

Accordingly, the control portion 11 detects the selection of the CM icon12 based on the input of the signal indicating the selection of theabove-described CM icon 12 and displays commercial information (a menu,the number of seats, an image inside the restaurant, the day's specials,or the like) of the restaurant that is included in the search resultinformation in the information display field SP through the displayportion 17. In addition, when the image data of the image inside therestaurant or the like is included in the commercial information, thecontrol portion 11 displays the image data in the information displayfield SP.

In addition, when the user touches a post browsing icon 13, theabove-described peripheral circuit outputs a signal indicating theselection of the post browsing icon 13 to the control portion 11.

Accordingly, the control portion 11 detects the selection of the postbrowsing icon 13 based on the input of the signal indicating theselection of the above-described post browsing icon 13 and displays thepost information (comment) written by other users, which is included inthe search result information, in the information display field SPthrough the display portion 17.

When there is a plurality of the above-described post information, thecontrol portion 11 displays the plurality of post information in theinformation display field SP, for example, in the order of the posteddate.

The post information can include image data. Thus, when the image dataof each food or the image data of the inside of the store that is takenby other users is included in the post information, the control portion11 displays the image data in the information display field SP throughthe display portion 17.

Then, for example, when detecting a signal indicating touch to be sledfrom the left end of the information display field SP toward the rightside in FIG. 5, the control portion 11 ends display in the informationdisplay field SP and displays the image of the above-describedrestaurant in the image display field MP as shown in FIG. 4 through thedisplay portion 17. In addition, for example, when detecting a signalindicating the selection of the positing browsing icon 13 in the middleof display of the post information in the information display field SP,the control portion 11 can be configured to end the display in theinformation display field SP or the display of the post information.

In addition, for example, when displaying the commercial information (orthe post information) after displaying the building information in theinformation display field SP, the control portion 11 can be configuredto display the commercial information together with the buildinginformation or to display the commercial information after removing thebuilding information from the information display field SP.

In addition, when detecting a signal indicating touch of the imageinside the store or the like that is displayed in the informationdisplay field SP, the control portion 11 displays an enlarged or reducedimage of the image inside the store in the information display field SP(or the image display field MP).

In addition, when the user touches a mail icon 14, the above-describedperipheral circuit outputs a signal indicating the selection of the mailicon 14 to the control portion 11.

Accordingly, the control portion 11 detects the selection of the mailicon 14 based on the input of the signal indicating the selection of theabove-described mail icon 14 and displays an input field IP as shown inFIG. 6 in a part of the image display field MP through the displayportion 17. Here, the input field IP is configured by a text field T towhich the user inputs information or the like and a touch screen-typekeyboard section K. This input field IP is used for the user to inputthe positing information, the name of the store, or the year to bedescribed later.

The user inputs the post information such as a food eaten by him, theatmosphere inside the store, or the quality of service served by a storestaff to the text field T of the input field IP by using the touchscreen-type keyboard section K. Then, as the user touches the mail icon14 again, the control portion 11 detects the selection of the mail icon14, sets the text information written in the text field T and the imagedata displayed in the image display field P together with the cameraidentification information and the building structure identificationinformation as the post information, and transmits a search requestsignal including the post information to the information search system 2through the communication portion 12.

Then, when receiving the above-described search request signal from thedigital camera 1, the server 21 writes new post information that hasbeen input to a post field of the building structure table of thedatabase 22 in association with the building structure identificationinformation. Alternatively, the server 21 can be configured to detectwhether there is the camera identification information, which isincluded in the search request signal, in the user registration table ofthe database 22 before writing the post information, which is written ina post write field G3, in the post field of the building structure tableof the database 22. In such a case, the above-described post informationcan be configured to be written in the post field of the buildingstructure table of the database 22 only in a case where registration ofthe camera identification information is detected.

Next, the control portion 11 detects whether a completion icon E isselected (Step S9).

Here, when the user touches the completion icon E, the above-describedperipheral circuit outputs a signal indicating the selection of thecompletion icon E to the control portion 11.

Then, when detecting the selection of the completion icon E based on theinput of the signal indicating the selection of the completion icon E,the control portion 11 completes the information searching process.

On the other hand, when the signal indicating the selection of thecompletion icon E is not input, and the selection of the completion iconE is not detected, the control portion 11 returns the process to StepS1, and the imaging process of the digital camera 1 and the informationsearching process are performed continuously.

When the browsing icon 11 is selected, the control portion 11 can beconfigured to control the search request signal to be transmitted to theinformation search system 2 so as to include the picked up image data.

Then, the server 21 can be configured to extract the characteristicamount from the image data by comparing peripheral image data of thebuilding structure information found in the building structure tablewith the transmitted image data by using the building structureidentification information of the building structure extracted from themap data by performing image recognition and to detect whether thesimilarity is equal to or more than a set value by comparing thecharacteristic amount extracted from the image data with that of thestored peripheral image data, whereby determining whether the image datacorresponds to the building structure of which the image is taken.

At this moment, when the building structure of the peripheral image dataand the building structure of the image data are determined not to besimilar to each other as the result of comparison, the server 21 can beconfigured to extract a building structure that is the secondly closestfrom the above-described position from the map data based on theposition information and the orientation information and to compare thecharacteristic amounts again, whereby the similarity between theperipheral image data of the newly extracted building structure and theimage data of which image is taken is evaluated.

In addition, the browsing icon 11 according to this embodiment,similarly to the above-described imaging button B1, can be set up in theimaging apparatus 1 as a browsing button (not shown). In such a case,the function of the browsing button is the same as that of theabove-described browsing icon 11. In addition, the imaging button B1according to this embodiment can be configured to serve as the browsingbutton by operating a switch portion not shown in the figure. In such acase, the browsing icon 11 or the browsing button is not needed.

<Information Search for Image after imaging by Using Digital Camera 1>

Next, the operation according to this embodiment will be described withreference to FIGS. 1, 7, and 4 to 6. FIG. 7 is a flowchart representingan operation example of an information searching process according tothis embodiment in a case where image data stored in the memory portion15 is read out after imaging an image of a building structure by usingthe digital camera 1, and the information search for the image data isperformed.

In description below, a case where a user searches for a diningrestaurant while strolling around a downtown street of Shinjuku, takesimages of image data of several restaurants, and then searches for arestaurant for dining out of the above-described restaurants isrepresented as an example.

When the user founds a restaurant having an exterior view for which aninformation search is desired to be performed and positions the digitalcamera 1 toward the restaurant having a favorite exterior view (StepS11) while strolling around the downtown street, the user presses downthe imaging button B1 (shutter) set up in the digital camera 1 (StepS12). Here, the control portion 11 displays the image data that is takenby the imaging portion 13, that is, the image data of the restaurant ofwhich the image is taken as shown in FIG. 4, in the image display fieldP of the display portion 17.

Next, as the user presses down the imaging button B1, the peripheralcircuit outputs a signal indicating press-down of the imaging button tothe control portion 11.

Then, the control portion 11 detects the press-down of the imagingbutton B1 based on the input of the signal indicating press-down of theimaging button B1, assigns the image identification information to theimage data of the restaurant of which the image is taken by the imagingportion 13, and stores the image data and the time of imaging in thememory portion 15 in association with the image identificationinformation (Step S13).

Then, the control portion 11 writes the position information acquired bythe GPS locator 14 and the orientation information acquired by theorientation sensor 16 at the time point when the imaging button B1 ispressed down in the memory portion 15 in association with the imageidentification information for storage (Step S14).

Next, when the user performs information collection of the restaurant ofwhich the image is taken, the user touches a read-out icon R of theimaging. Accordingly the peripheral circuit outputs a signal indicatingthe selection of the read-out icon R to the control portion 11.

Here, the control portion 11 determines whether an image is stored inthe memory portion 15 based on the input of the signal indicating theselection of the read-out icon R (Step S15).

At this moment, when the signal indicating the selection of the read-outicon R is input, the control portion 11 detects the selection of theread-out icon R, and the process proceeds to Step A16 for starting aninformation search. In addition, the read-out icon R can be a read-outbutton (not shown). In such a case, it is preferable that the read-outbutton, similarly to the imaging button B1, is set up in the main bodyof the digital camera 1.

On the other hand, when the signal indicating the selection of theread-out icon R has not been input or the image is not stored in thememory portion 15, the control portion 11 returns to Step S11 forperforming a new imaging process.

Then, when receiving the signal indicating the selection of the icon Rfrom the peripheral circuit by the user touching the read-out icon R asinput, the control portion 1I sequentially reads out image data from thememory portion 15 in the order of imaging in a time series (Step S16)and displays the image data corresponding to a predetermined number setin advance as thumbnail images (or each image data) in the image displayfield MP through the display portion 17.

In addition, in a case where the thumbnail images are not fit on onepage (or one thumbnail image is displayed each time), when the usertouches the image displayed in the image display field MP so as to besled in a predetermined direction, a thumbnail image of the previouspage or the next page is displayed in the image display field MP. Atthis moment, the control portion 11 displays thumbnail images of theprevious page or the next page in the image display field MP through thedisplay portion 17 in a case where the signal indicating touch of theimage displayed in the image display field MP so as to be sled in thepredetermined direction is detected.

Next, when the user selects an interesting image of a restaurant, forexample, from among the thumbnail images by touching the image andadditionally touches the browsing icon 11, the peripheral circuitoutputs the signal indicating the selection of the browsing icon 11together with the selected image identification signal to the controlportion 11.

Accordingly, when receiving a signal indicating press-down of thebrowsing icon 11 as input, the control portion 11 reads out the positioninformation and the orientation information corresponding to the imageidentification information from the memory portion 15 by using the imageidentification information of the selected image data and transmits asearch request signal including the camera identification information,the position information, and the orientation information to theinformation search system 2 through the communication portion 12 (StepS5).

The processes of the following Steps S6 to S8 are the same as those ofSteps S6 to S8 represented in FIG. 3, and thus description thereof isomitted here.

Next, the control portion 11 detects whether the completion icon E isselected (Step S18).

At this moment, the control portion 11 receives a signal indicating theselection of the completion icon E as input from the peripheral circuitby the user touching the completion icon E, the control portion 11 endsthe browsing process. On the other hand, when not receiving the signalindicating the selection of the completion icon E as input, the controlportion 11 returns the process to Step S16, and the process of selectingan image from among the thumbnail images is continued.

<Information Search Using Store Information Input to Digital Camera 1>

Next, the operation according to this embodiment will be described withreference to FIGS. 1, 8, and 4 to 6. FIG. 8 is a flowchart representingan operation example of the information searching process according tothis embodiment for a case where an information search is desired to beperformed by user's inputting a store name to the digital camera 1 for arestaurant or the like having the input store name from the buildingstructure table, represented in FIG. 2, corresponding to the store namethat is stored in the database 22.

In the following description, for example, is a case where theinformation on a restaurant in Harajuku from a known person is checked.

When the user selects a search icon 15 by touching the search icon 15,the peripheral circuit outputs a signal indicating the selection of thesearch icon 15 to the control portion 11.

Accordingly, the control portion 11 detects the selection of the searchicon 15 based on the input of the signal indicating the selection of thesearch icon 15 and displays the input field IP as shown in FIG. 6 in apart of the image display field MP through the display portion 17.

Then, the user writes the store name of the restaurant desired to besearched for in the text field T of the input field IP by using thetouch screen-type keyboard section K (Step S21) and touches the searchicon 15 again (Step S22).

Accordingly, the peripheral circuit outputs text data (character data)of the store name that has been input to the text field T together withthe signal indicating the selection of the search icon 15 to the controlportion 11.

Then, the control portion 11 detects the selection of the search icon 15based on the input of the signal representing the selection of thesearch icon 15, reads in the text data of the store name that has beeninput to the text field T. and transmits the search request signaltogether with the camera identification information to the informationsearch system 2 through the communication portion 12 (Step S23).

Next, when receiving the search request signal, the server 21 reads outthe information (the store information and each information representedin the table represented in FIG. 2) on the building structure (store)corresponding to the store name from the building structure table of thedatabase 22 (Step S24).

After reading in the store information, the server 21 transmits theacquired store information to the digital camera 1 (Step S25).

When receiving the store information through the communication portion12, the control portion 11 displays building information in theinformation display field SP together with displaying the image data ofthe periphery of the store that is included in the building structureinformation in the image display field MP represented in FIG. 5 throughthe display portion 17 (Step S26). For example, the type (Chinese,Japanese, French, Italian, or the like) of the restaurant, the telephonenumber, and the like are displayed.

In addition, when the user selects a CM icon 12 by touching the CM icon12, the control portion 11 detects the selection of the CM icon 12 anddisplays commercial information (a menu, a chef's today's special, orthe like) of the restaurant included in the search result information inthe information display field SP through the display portion 17.

In addition, when the user selects the post browsing icon 13, bytouching the post browsing icon 13, the peripheral circuit outputs asignal representing the selection of the post browsing icon 13 to thecontrol portion 11.

The control portion 11 detects the selection of the post browsing icon13 based on the input of the signal indicating the selection of the postbrowsing icon 13 and displays the post information written by otherusers which is included in the search result information in the displayfield SP through the display portion 17.

When there is a plurality of the post information, the control portion11 sequentially displays the plurality of the post information in theinformation display field SP through the display portion 17. Inaddition, when image data is also included in the post information, thecontrol portion 11 displays the image data in the information displayfield SP through the display portion 17.

Then, the control portion 11 detects whether the completion icon E hasbeen selected (Step S27). The detection of the selection of thecompletion icon E is the same as the process of Step S9 in “<Searchingfor Information on Imaging By Digital Camera 1>”, and thus, descriptionthereof is omitted here.

At this moment, when receiving the signal indicating the selection ofthe completion icon E as input by user's touching the completion icon Efrom the peripheral circuit, the control portion 11 ends the browsingprocess. On the other hand, when not receiving the signal indicating theselection of the completion icon E, the control portion 11 returns theprocess to Step S21 so as to continue to perform an information searchfor the store.

<Charging Process for Store Registered in Database 22>

In a case where the above-described building structure is a store(including a restaurant, or a company), the building structure table ofthe database 22 can be configured such that discount coupon data isstored for each store.

In addition, the server 21 transmits the search result information towhich the discount coupon information is added to the digital camera 1.

When the user goes dining or shopping by using the discount couponinformation, the server 21 detects whether the user has dined or shoppedbased on whether the discount coupon information has been used. Forexample, in a case where the above-described coupon information is a barcode (including a two-dimensional bar code) or the like displayed in theinformation display field SP of the digital camera 1, a reading portionof the store reads out the bar code and transmits use informationindicating the use of the discount coupon information together with thecamera identification information, which is transmitted from the digitalcamera 1, and shopping (or dining) charge to the information searchsystem 2.

Accordingly, when receiving the use information, the server 21 reads outthe history information corresponding to the camera identificationinformation added to the use information from the database 22 anddifferently charges the store with an information providing feedepending on the level of information use out of a level in which thestore is used by performing an information search, a level in which thestore is used by viewing the provided information on the store, and alevel in which the store is used by viewing the post information, thatis, the degree of use of the information of the database 22. Inaddition, the server 21 writes the level of the information search andthe history of the use of the discount coupon information for eachcamera identification information in the history table that is stored inthe database 22.

This bar code includes the building structure identification informationthat represents a store. In the database 22, a charge table, in whichthe history of charging and an accumulated value of chargingcorresponding to each store are stored in association with the buildingconstruction identification information, is set up.

Then, the server 21 writes the amount of usage and a chargecorresponding to the level of information use as the history in thecharge table in association with the building structure identificationinformation and adds the new charge to the accumulated value so as toupdate the accumulated value.

<Process of Adding Points to User>

Each time the user uses the discount coupon information at the store,the server 21 calculates points corresponding to the amount of usage(for example, calculated by multiplying the amount of usage by a pointcoefficient) for a case where the discount coupon information is usedand accumulates the calculated points in the user registration table ofthe database 22 in association with the camera identificationinformation.

In addition, the server 21 accumulates points corresponding to a valueset in advance for a user who has transmitted the post information inthe user registration table in association with the cameraidentification information.

The above-described points can be used, instead of money, together withthe discount coupon information in paying the charge.

<Display Order of Post Information>

Here, the server 21 can be configured to allow the digital camera 1 totransmit a nickname and a title as the post information together withthe post identification information in the initial stage.

Then, the control portion 11 initially displays only a plurality ofnicknames and a plurality of titles in the information display field SP.Then, the user selects the nickname and the title of the postinformation desired to be known from among the plurality of the nicknames and the titles displayed by touching the nick name and the titleof the post information. Accordingly, the peripheral circuit transmitsthe signal indicating of the selection of the nickname and the title andthe post identification information of the selected post information tothe control portion 11.

Accordingly, the control portion 11 transmits a post informationtransmission request together with the post identification informationto the information search system 2.

Accordingly, the server 21 transmits the text data (character data) andthe image data (the entire post information) corresponding to thetransmitted post identification information to the digital camera 1.

Here, the server 21 assigns the positing identification information thatis used for identification for each of the post information and writesthe post information in the user registration table in association withthe camera identification information for storage.

In addition, in this embodiment, the control portion 11 can beconfigured to display the text data in the information display field SPand display the image data in the image display field MP when receivingthe text data and the image data from the information search system 2through the communication portion 12.

In addition, the server 21 searches for the post identificationinformation corresponding to the referred post information from the userregistration table and increments the number of references for thecamera identification information corresponding to the postidentification information in the user registration table.

Then, in the initial display stage of the nickname and the title inwhich the user can select the post information, the server 21 transmitsa display list, in which the nicknames and the titles are displayed inthe descending order of the number of times of the selection of the postinformation, that is, the nickname and the title having more number oftimes of references comes first to be displayed in the informationdisplay field SP to the digital camera 1.

Accordingly, the control portion 11 sequentially displays the nicknamesand the titles in the information display field SP in accordance withthe display list.

<Process of Displaying Past Image Data>

In addition, according to this embodiment, the database 22 can beconfigured to have a past image table in which image data acquired byphotographing buildings and landscapes at each latitude and eachlongitude are photographed for each year is stored in association withthe latitude and the longitude.

When the user selects the past image icon 16 by touching the past imageicon 16, the above-described peripheral circuit outputs a signalindicating the selection of the past image icon 16 to the controlportion 11.

Accordingly, the control portion 11 detects the selection of the pastimage icon 16 based on the input of the signal indicating the selectionof the past image icon 16 and displays an input field IP as representedin FIG. 6 in a part of the image display field MP through the displayportion 17.

Then, after writing the year (for example, the dominical year) in thetext field T by using the touch screen-type keyboard section K, the usertouches the past image icon 16 (or the browsing icon 11).

Accordingly, the peripheral circuit transmits the text data of the yeartogether with the signal indicating the selection of the past image icon16 to the control portion 11.

When detecting the signal indicating the selection of the past imageicon 16, the control portion 1I reads in the text data representing theyear written in the text field T. After reading in the data for theyear, the control portion 11 transmits the past image search requesttogether with the read year, the position information, the orientationinformation, and the camera identification information to theinformation search system 2.

Next, in the information search system 2, the server 21 selects the pastimage table corresponding to the latitude and the longitude on the basisof the position information of the building structure that is receivedfrom the digital camera 1 and is displayed in the image display field P.Then, the server 21 reads in the image data corresponding to theorientation information and the year in the selected past image tableand transmits the read image data to the digital camera 1. The digitalcamera 1 displays the image data corresponding to the year received fromthe server 21 in the image display field MP through the display portion17. In addition, in a case where the information or the like for theread-out image is added, the server 21 transmits the added informationtogether with the read-out image data to the digital camera 1.

Accordingly, the user can acquire information indicating that there is aspecific building structure for each year in the past or there is onlythe landscape without any building structure for each year in the past.

Here, when the past image table of the corresponding latitude and thelongitude cannot be found, the server 21 searches for the past imagetable that is located at the latitude and the longitude in the directionof the orientation information and is closest to the latitude and thelongitude and extracts the image data therefrom.

In addition, for an era without a photo, a landscape that had lasted inthe era or image data of CG (computer graphics) generated by imaginationcan be stored for each year in the past image table of the memoryportion 15 in association with the year, instead of the image data of aphoto.

An embodiment relates to an imaging apparatus that has a function ofacquiring information on an imaging position of a subject andassociating the subject and the information on the imaging position atthe time of imaging the subject, and an information acquisition systemthat acquires information on the subject by using the information on theimaging position.

In the embodiment, the orientation information (for example, an azimuthangle) for acquiring the direction in which the user photographs thesubject at the photographing position can be acquired.

In the embodiment, the information on the subject that is photographedby the user, for example, the type of a building structure can beacquired easily.

In the embodiment, not only the position information of the imagingposition in which the user photographs a subject but also theorientation information representing the direction, in which the usertakes an image of the subject from the position represented by theposition information that is acquired by the position informationacquisition portion, can be acquired. Accordingly, the orientation ofthe photographed subject, that is, the direction in which the userdisposes the imaging apparatus can be determined.

In the embodiment, the position information and the orientationinformation obtained by imaging a subject can be acquired. Accordingly,the information on a building structure as a subject of which the imageis taken by the user by using the imaging apparatus can be easilyacquired by searching a database by using the position information andthe orientation information acquired by imaging.

1.-4. (canceled)
 5. A portable information device configured to be inwireless communication with an information search server via aninformation communication network, the portable information devicecomprising: a body; an image sensor that is provided in the body and isconfigured to image a subject and output image information regarding animage of the subject; a locator that is provided in the body and isconfigured to output position information; a transmitter that isprovided in the body and is configured to transmit the image and theposition information to the information search server and receive searchresult information from the information search server; a memory that isprovided in the body; a touch panel display that is provided in thebody; and a controller that is provided in the body and is configuredto: in response to a touch operation, display a plurality of imagesstored in the memory on the touch panel display, in response to a touchoperation to select a building structure image as a search target amongthe plurality of displayed images, execute a search about the selectedbuilding structure image, display a result of the search as searchresult information on the touch panel display along with the targetstructure image, the search result information including relevant imageinformation about the search target building structure and relevant textinformation about the relevant image information, the search resultinformation being based on the position information corresponding to thetarget structure image and a result of an image recognition for thetarget structure image; and in response to another touch operation whilethe search result information is displayed on the touch panel display,display additional information about the search result information onthe touch panel display.
 6. The portable information device according toclaim 5, wherein the search result information is output, based on adetermination of the search target building structure, by comparing acharacteristic amount of the target structure image with acharacteristic amount of relevant structure image extracted from adatabase of the information search server by using the positioninformation.
 7. The portable information device according to claim 5,further comprising: an orientation sensor, wherein the positioninformation is used along with orientation information from theorientation sensor, wherein the controller is configured to control thetransmitter to transmit the orientation information along with theposition information to the information search server, and the searchresult information is based on the position information corresponding tothe target structure image, the orientation information corresponding tothe target structure image and a result of an image recognition for thetarget structure image.
 8. The portable information device according toclaim 5, wherein the additional information includes text informationabout the search target building structure and image information aboutthe search target building structure.
 9. The portable information deviceaccording to claim 5, wherein the search target building structure is astore, and the additional information includes commercial information,the commercial information including an image captured inside of thestore and product information.
 10. The portable information deviceaccording to claim 5, wherein the search target building structure is astore, and the additional information includes posted information aboutthe store.
 11. An information acquisition system comprising: an imagerthat captures an image of a search target and outputs image data; aposition sensor that acquires position information of an imagingposition; a memory that stores the image data and the positioninformation; a display that displays images and text, and detects atouch operation as an instruction operation by a user; an informationretrieval server that performs information retrieval based on the imagedata from the imager and the position information from the positionsensor; and a computer having a first mode and a second mode, one ofwhich is selected in response to the touch operation by the user,wherein in the first mode, (a) a target image is displayed on thedisplay based on image data of a target captured via the imager andrecognized by the user, (b) in response to a touch operation by theuser, target data of a search target is transmitted to the informationretrieval server, the target data including data of the target image andposition information, the position information being acquired via theposition sensor and corresponding to the target image, (c) first searchresult information is displayed so that both the target image andsearched image data are displayed on the display together, the firstsearch result information including the searched image data and textdata as a primary search result received from the information retrievalserver based on the target image and the position information, and (d)second search result information transmitted from the informationretrieval server is displayed on the display based on a further searchbased on signals transmitted to the information retrieval server, thefurther search being executed in response to a touch operation by theuser in a state in which the searched image data is displayed on thedisplay together with the target image, and in the second mode, (e) aplurality of images stored in the memory are displayed on the display,(f) in response to a touch operation by the user to select a targetimage among the plurality of displayed images, information retrieval isperformed based on the selected target image and position informationcorresponding to the selected target image, (g) third search resultinformation is displayed on the display together with the selectedtarget image, the third search result information including searchedimage data and text data as a primary search result from the informationretrieval server, and (h) fourth search result information transmittedfrom the information retrieval server is displayed on the display basedon a further search, the further search being executed in response to atouch operation by the user in a state in which the third search resultinformation is displayed on the display together with the selectedtarget image.
 12. The information acquisition system according to claim11, wherein the information retrieval server is configured (1) tocompare amount of features by image recognition between the capturedimage data of the search target and stored image data in a database and(2) to determine whether the captured image data corresponds to thesearch target based on the comparison result.
 13. An imaging devicecomprising: an imager; a position sensor; a memory; a transmitter; adisplay; and a controller configured to control the imager, the memory,the transmitter, and the display and having a first mode and a secondmode, one of which is selected in response to a touch operation by auser, wherein in the first mode, (a) a subject image is displayed on thedisplay based on image data of a subject captured via the imager andrecognized by the user, (b) in response to a touch operation by theuser, subject data of a search target is transmitted to an informationretrieval server via the transmitter, the subject data including data ofthe subject image and position information, the position informationbeing acquired via the position sensor and corresponding to the subjectimage, (c) first search result information is displayed so that both thetarget image and searched image data are displayed on the displaytogether, the first search result information including the searchedimage data and text data as a primary search result from the informationretrieval server based on the data of the subject image and the positioninformation, and (d) second search result information transmitted fromthe information retrieval server via the transmitter is displayed on thedisplay based on a further search based on signals transmitted betweenthe transmitter and the information retrieval server, the further searchbeing executed in response to a touch operation by the user in a statein which the searched image data is displayed on the display togetherwith the held subject image.
 14. The imaging device according to claim13, wherein in the second mode, (e) a plurality of subject images storedin the memory are displayed on the display, (f) in response to a touchoperation by the user to select a subject image among the plurality ofdisplayed subject images, subject data of a search target is transmittedto the information retrieval server via the transmitter, the subjectdata including data of the selected subject image and positioninformation corresponding to the selected subject image, (g) thirdsearch result information is displayed on the display together with theselected subject image, the third search result information includingsearched image data and text data as a primary search result from theinformation retrieval server based on the data of the selected subjectimage and the position information, and (h) fourth search resultinformation transmitted from the information retrieval server via thetransmitter is displayed on the display based on a further search basedon signals transmitted between the transmitter and the informationretrieval server, the further search being executed in response to atouch operation by the user in a state in which the third search resultinformation is displayed on the display together with the selectedsubject image.